“Experience as a whole is not something that anyone can be spared”. — Hans-Georg Gadamer

Data Driven is a Car Crash?

Thinking about -and operating in -an unknowable future is hard.

But being data driven sure helps cover that. Data is the ultimate form of competitive advantage. It enables to build accurate models of customer-competitor-market behaviour which can ‘inform’ strategic choices.

Data says ‘let there be light!’ and — lo- there is light.

Or is there?

Humans beings love certainty. We need it to function. Data helps us test experiments that we create on the basis of our interactions with…well…yes… data.

Theoretically, as long as you can define the scale of the experiment, the metrics used to track it and the functions — input to output relationships- it fulfils, you can build effective predictive- causative- models to navigate the unknown.

Of course at some scales this appears to be incorrect. A quantum universe might, for instance, abhor certainty. Admittedly, in traditional quantum mechanics causality is always definite. In 2012 Oreshkov et al developed a general framework for the study of correlations between quantum events without the assumption of a predefined and definite causal order between them. Through these correlations, we can then consider their causal structure. Using such a formalism, it is possible to obtain correlations between events that are incompatible with an underlying causal order.

The event environment of the events is represented by a mathematical object called a process matrix. A process matrix is the resource that allows the events to be correlated in a particular way, compatible or not within a definite causal structure.

So, we can see that scale is a critical. What is true at one scale may not be true at another. It is an — often unrecognised- component of understanding in disciplines as diverse as cognition, investment, science, economics, politics and history.

This is not simply because of limits to human representational capacity. None of us has ever seen a quark. Nor can we- as yet- act upon one directly. So, data rests upon both representational accuracy and actionable capacity. All the data in the Universe is not going to help you unless you can act upon it.

Now, currently, we have a model which focuses on accurate representation where a collection of data is held to be representative of one data point or data set. This is cool. It is the very touchstone of academic — and by extension- commercial research. Build a theory, design the research which will test it, carry out and then validate the research. And to hell with P-hacking and significance shopping.

On the web, the results of this are manageable, because the actions that this data represents are so limited. Click, click, click, click. And these actions underpin limited states and properties. As well as limited models. Because digital technology-defined as it was by Shannon- is about reducing costs in a much wider way, rather than improving outcomes.

Of course, it is counter-intuitive to believe that digital transformation is not about improving our lives. But is it?

Because, if you are as data driven as you hope and claim to be, how do you choose the scale which defines how you approach the problem? How do you select the metrics? And how do you model the functions?

And, of course, the illusion that research is unaffected by politics throughout every aspect of its processes, is — sadly — delusional. And if you don’t believe me, listen to a senior Google executive a couple of weeks ago telling companies that actually make and sell things- that they ‘had’ to give up their data to Google’s digital transformation.

Understandably, perhaps, they were reluctant to do so.

Oh, sure, it is useful to have data driven as an -increasingly fragile- mantra which is chanted with desperation from webinar to think-piece in the WSJ. It may drive sales. But should it drive strategy?

But what’s the real upshot apart from a general sense of self-satisfaction that you have your finger (don’t use your thumb, it has a pulse of its own) on the pulse of reality? When reality is increasingly frail? If you have no scale, the amount and nature of the data you have is…well…useless.

Thinking Big or Thinking Right?

The idea that data doesn’t work is…well…terrifying. It requires an alternative to even function. And how’s that going to work? If we don’t use data, do we use belief? Even justified true belief? Or do we develop a vision? And what would it say?

Because visions might be just a data driven as any candlestick chart or Tableau visualisation?

Today, very few people would accept the ‘vision’ that the world is about to end through a supernatural act forecast by prophecy. Many more would accept that it is about to end as a result of existential threats such as climate change or nuclear war. Sure, property prices in San Francisco might be falling and bunker building in New Zealand might be increasing, but what scale should be used to measure the real impact of such ‘facts’? And what metrics and what functions? And what models do we derive from them or use to inform them?

Uniting Data and Vision

Despite the ongoing support of commentators on the far right, the Judeo-Christian four horsemen type apocalypse, at least as it is literally understood, is normally counted as a creed which cannot be taken seriously.

But this has not always been true. Between 1500 and 1800 many of the most creative minds in that part of the ‘civilised’ world which shared some commonality of belief systems. believed that they were living in the latter days of the world and faced the end of human history. Like Fukuyama in the 90s.

But visions of an apocalypse are often a signal of dissatisfaction. They are ‘used’ — consciously or otherwise- to provoke change.

The vision of apocalyptic horseman underwrote- in part- the Reformation in the sixteenth century, the English Civil War in the seventeenth century, and the American Revolution in the eighteenth century.

And this provided a crucial catalyst in the emergence of liberal values, political democracy, and the creation of modern science in the enlightenment.

The early modern period was not the first time that the idea of an apocalypse penetrated the Western intellect and redefined it. That had happened once before during the Intertestamental years (150 BCE–200 CE). At this time apocalyptic expectations permanently transformed the religious landscape and, eventually -but consequentially- the political and economic landscape as well.

But in cognition, investment, science, economics, politics and history, we need to unite data and vision to make data ‘work’.

Because data breeds unintended consequences.

Living with Unintended Data Consequences

Between 1500 and 1800 the second encounter with the apocalypse created the modernity which -in turn- created the context in which we now live. During that second great encounter, such expectations played a central role in the emergence of secular culture — arguably the most important achievement of the postmediaeval West.

This is, of course, ironic. A deeply fixed set of religious ideas about God being annoyed proved instrumental in enabling people in developing alternatives to a religious perspective. And led- as we shall see below- to the current economic crisis, whether you believe in Covid-19. Or not.

Secular categories, initially, arose less from the rejection of religion than through the dynamics and tensions within religion itself. Religion failed to provide a coherent, believable, fair working explanation of how the world worked and how individuals could work within it.

And while the apocalypse provided an immediate religious “stick” and “carrot” made visible, it had unexpected consequences which led us to another set of lenses through which we contemplate- and understand- the world.

These three lenses enable an effective operational model of reality.

1. The dependability lens offers a relatively stable construct called human nature exists across history and other forms of situational context.

2. The predictability lens offers us an understanding of how we can overcome the influence of contexts on concepts such as belief, knowledge, and understanding are contingent and relatively weak in the sense that they could in principle be resisted by the humans involved

3. The consistency lens allows us to assume that behavioural differences can largely be described as a contrast resulting from the use of a faculty of context-independent reasoning with which all humans are constitutionally equipped by nature

So, what might be the unexpected impact of the explosion in the secular data driven world where anyone can run experiments? With descriptive, evaluative, explanatory predictive, reproducible by independently verifiable experiment (DEEPRIVE) models which have no ‘social’ anchor apart from social media and online validity.

So that both the experiment and the data that confirms its underpinning correspondence with the reality it constructs

And how do you want to shape and respond to it?

With data?

Or do you need something more?

Because AI and technology will not provide an escape route. Uniting ‘the data’ and ‘the vision’ is difficult. They are intercausative. That is, in such cases- causation is bidirectional. Vision- experiment- defines the data collection. And the data defines the vision.

And both of these rely upon more than linguistic or mathematical symbols- which are, after all- just another technology.

Thinking Bigger

Of course, it depends whether you accept the premise. The apocalypse gave us the Enlightenment. And the Enlightenment gave us the idea of the individual human as an independent thinking entity. Freed from superstition and religious dogma, the individual was now in almost complete control of her innate and acquired capabilities.

Of course, this raised a new problem- the loss of moral certitude. The Kantian syllogism — it is moral to be rational and rational to be moral- tried to solve this problem. But the young discipline of economics seized on this newly created autonomous agent and assigned that concept the task of rendering rational — self- interested — choices.

Such choices revealed true value: the rational individual could not be wrong about her choices. Individualism was now welded to acquisitive behaviour. Weber’s The Protestant Ethic and the Spirit of Capitalism provided a theoretical base for an emergent culture of enterprise, striving and advantage through competition. This was further reinforced by Darwinian models of evolution which supported individual survival .

By the middle of the twentieth century, economics had become the primary religion in a modern industrial state which had to provide ever- increasing living standards for the growing number of citizens upon which it relied for tax revenues in the same way the young lady of Riga relied on her source of transportation. And certain forms of economic data were used to drive social and political decision making and the belief systems which underpinned these decisions.

Unemployment rates, inflation rates, changes in the stock market became the contemporary equivalent of visions from God, ancient astrological sightings and making sacrifices of one’s children.

And furthermore, as with any guidance system- spiritual or secular- there were schisms and movements industrial, financial, or managerial capitalism. The Great Financial Crash of 2007– 2009 delivered a surprising destructive shock to large swathes of the population in western Europe and the United States. The fear, suffering and anger produced by that disruption are likely to be compounded and its effects widened by the so-called Covid19 Recession.

Mounting despair, political revolts and populist governments, coupled with massive migration have become a threat to the economic belief systems which defined life in the West since around the end of World War II. The political and economic classes, regardless of party affiliation, are now bereft of ideas.

We need to think bigger.

Not just in terms of how we scale, but also in terms of who we include. And how we include them.